正文

68 - Quick introduction to Support Vector Machines -SVM

sklearn.svm.SVC — scikit-learn 1.2.0 documentation

C-Support Vector Classification.

  • The implementation is based on libsvm. The fit time scales at least quadratically with the number of samples and may be impractical beyond tens of thousands of samples. For large datasets consider using LinearSVC or SGDClassifier instead, possibly after a Nystroem transformer.

    • 该实现基于 libsvm。拟合时间至少与样本数量成二次方比例，超过数万个样本可能不切实际。对于大型数据集，考虑使用 LinearSVC 或 SGDClassifier，可能在 Nystroem transformer 之后。

  • The multiclass support is handled according to a one-vs-one scheme.

    • 多类支持根据一对一方案进行处理。

  • For details on the precise mathematical formulation of the provided kernel functions and how gamma, coef0 and degree affect each other, see the corresponding section in the narrative documentation: Kernel functions.

    • 有关所提供内核函数的精确数学公式以及gamma、coeff0和degree如何相互影响的详细信息，请参阅叙事文档中的相应部分：内核函数。

Read more in the User Guide.

**Parameters 参数: **

C: float, default=1.0

  • Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. The penalty is a squared l2 penalty.
    • 正则化参数。正则化的强度与C成反比。必须严格为正。惩罚是l2的平方惩罚。

kernel: {‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’} or callable, default=’rbf’

  • Specifies the kernel type to be used in the algorithm. If none is given, ‘rbf’ will be used. If a callable is given it is used to pre-compute the kernel matrix from data matrices; that matrix should be an array of shape (n_samples, n_samples).
    • 指定要在算法中使用的内核类型。如果没有给出，将使用“rbf”。如果给定了一个可调用函数，则它用于从数据矩阵中预计算内核矩阵；该矩阵应该是形状“（n_samples，n_samples）”的数组。

degree: int, default=3

  • Degree of the polynomial kernel function (‘poly’). Must be non-negative. Ignored by all other kernels.
    • 多项式核函数的次数（‘poly’）。必须为非负。被所有其他内核忽略。（只在内核类型为 poly 时有效）

gamma：{‘scale’, ‘auto’} or float, default=’scale’*

  • Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’.

    • “rbf”、“poly”和“sigmoid”的核系数。

  • if gamma='scale' (default) is passed then it uses 1 / (n_features * X.var()) as value of gamma,

  • if ‘auto’, uses 1 / n_features

  • if float, must be non-negative.

Changed in version 0.22: The default value of gamma changed from ‘auto’ to ‘scale’.

coef0: float, default=0.0

  • Independent term in kernel function. It is only significant in ‘poly’ and ‘sigmoid’.
    • 核函数中的独立项。它只在“poly”和“sigmoid”中有意义。

shrinking: bool, default=True

  • Whether to use the shrinking heuristic. See the User Guide.
    • 是否使用收缩启发式。

probability: bool, default=False

  • Whether to enable probability estimates. This must be enabled prior to calling fit, will slow down that method as it internally uses 5-fold cross-validation, and predict_proba may be inconsistent with predict. Read more in the User Guide.
    • 是否启用概率估计。这必须在调用“fit”之前启用，这会减慢该方法的速度，因为它内部使用5倍交叉验证，而且“predict_proba”可能与“predict”不一致。

tol: float, default=1e-3

  • Tolerance for stopping criterion.
    • 停止标准公差。

cache_size: float, default=200

  • Specify the size of the kernel cache (in MB).
    • 指定内核缓存的大小（以MB为单位）。

class_weight: dict or ‘balanced’, default=None

  • Set the parameter C of class i to class_weight[i]*C for SVC. If not given, all classes are supposed to have weight one. The “balanced” mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as n_samples / (n_classes * np.bincount(y)).
    • 对于SVC，将类i的参数C设置为class_weight[i]*C。如果没有给出，所有的课程都应该有一个权重。“平衡”模式使用y值自动调整与输入数据中的类频率成反比的权重，如n_samples/(n_classes*np.bincount(y))。

verbose: bool, default=False

  • Enable verbose output. Note that this setting takes advantage of a per-process runtime setting in libsvm that, if enabled, may not work properly in a multithreaded context.
    • 启用详细输出。注意，此设置利用了libsvm中的每进程运行时设置，如果启用该设置，则在多线程上下文中可能无法正常工作。

max_iter: int, default=-1

  • Hard limit on iterations within solver, or -1 for no limit.
    • 解算器内迭代的硬限制，或-1表示无限制。

decision_function_shape: {‘ovo’, ‘ovr’}, default=’ovr’

Whether to return a one-vs-rest (‘ovr’) decision function of shape (n_samples, n_classes) as all other classifiers, or the original one-vs-one (‘ovo’) decision function of libsvm which has shape (n_samples, n_classes * (n_classes - 1) / 2). However, note that internally, one-vs-one (‘ovo’) is always used as a multi-class strategy to train models; an ovr matrix is only constructed from the ovo matrix. The parameter is ignored for binary classification.

Changed in version 0.19: decision_function_shape is ‘ovr’ by default.

New in version 0.17: decision_function_shape=’ovr’ is recommended.

Changed in version 0.17: Deprecated decision_function_shape=’ovo’ and None.

break_ties: bool, default=False

If true, decision_function_shape='ovr', and number of classes > 2, predict will break ties according to the confidence values of decision_function; otherwise the first class among the tied classes is returned. Please note that breaking ties comes at a relatively high computational cost compared to a simple predict.

New in version 0.22.

random_state: int, RandomState instance or None, default=None

  • Controls the pseudo random number generation for shuffling the data for probability estimates. Ignored when probability is False. Pass an int for reproducible output across multiple function calls. See Glossary.
    • 控制伪随机数的生成，以便对概率估计的数据进行混洗。当“probability”为False时忽略。在多个函数调用之间传递一个int以获得可复制的输出。

**Attributes 属性: **

class_weight_: ndarray of shape (n_classes,)

  • Multipliers of parameter C for each class. Computed based on the class_weight parameter.
    • 每个类的参数 C 的乘数。基于 class_weight 参数计算。

classes_: ndarray of shape (n_classes,)
The classes labels.

coef_: ndarray of shape (n_classes * (n_classes - 1) / 2, n_features)

  • Weights assigned to the features when kernel=“linear”.
    • 当 kernel=“linear”时，分配给特征的权重。

dual_coef_: ndarray of shape (n_classes -1, n_SV)

  • Dual coefficients of the support vector in the decision function (see Mathematical formulation), multiplied by their targets. For multiclass, coefficient for all 1-vs-1 classifiers. The layout of the coefficients in the multiclass case is somewhat non-trivial. See the multi-class section of the User Guide for details.
    • 决策函数中支持向量的对偶系数（参见数学公式）乘以其目标。对于多类别，所有 1-vs-1 分类器的系数。在多类情况下，系数的布局有些不平凡。有关详细信息，请参阅《用户指南》的多类部分。

fit_status_: int

  • 0 if correctly fitted, 1 otherwise (will raise warning)
    • 如果安装正确，则为 0，否则为 1（将发出警告）

intercept_: ndarray of shape (n_classes * (n_classes - 1) / 2,)

  • Constants in decision function.
    • 决策函数中的常量。

n_features_in_: int

  • Number of features seen during fit.
    • 装配过程中看到的特征数量。

New in version 0.24.

feature_names_in_: ndarray of shape (n_features_in_,)

  • Names of features seen during fit. Defined only when X has feature names that are all strings.
    • 装配过程中看到的特征名称。仅当 X 具有全部字符串的要素名称时才定义。

New in version 1.0.

n_iter_: ndarray of shape (n_classes * (n_classes - 1) // 2,)

  • Number of iterations run by the optimization routine to fit the model. The shape of this attribute depends on the number of models optimized which in turn depends on the number of classes.
    • 优化例程为拟合模型而运行的迭代次数。该属性的形状取决于优化的模型数量，而优化的模型又取决于类的数量。

New in version 1.1.

support_: ndarray of shape (n_SV)

  • Indices of support vectors.
    • 支持向量索引。

support_vectors_: ndarray of shape (n_SV, n_features)
Support vectors.

n_support_: ndarray of shape (n_classes,), dtype=int32
Number of support vectors for each class.

probA_: ndarray of shape (n_classes * (n_classes - 1) / 2)
Parameter learned in Platt scaling when probability=True.

  • 当概率为 True 时，在 Platt 缩放中学习的参数。

probB_: ndarray of shape (n_classes * (n_classes - 1) / 2)
Parameter learned in Platt scaling when probability=True.

shape_fit_: tuple of int of shape (n_dimensions_of_X,)
Array dimensions of training vector X.

68b - SVM vs. Random Forest for image segmentation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
import numpy as np
import cv2
import pandas as pd
 
img = cv2.imread('images/Train_images/Sandstone_Versa0000.tif')
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)  
img2 = img.reshape(-1)
df = pd.DataFrame()
df['Original Image'] = img2

# Generate Gabor features
num = 1
kernels = []
for theta in range(2):
    theta = theta / 4. * np.pi
    for sigma in (1, 3):
        for lamda in np.arange(0, np.pi, np.pi / 4):
            for gamma in (0.05, 0.5):  
                gabor_label = 'Gabor' + str(num)
                ksize=9
                kernel = cv2.getGaborKernel((ksize, ksize), sigma, theta, lamda, gamma, 0, ktype=cv2.CV_32F)    
                kernels.append(kernel)
                fimg = cv2.filter2D(img2, cv2.CV_8UC3, kernel)
                filtered_img = fimg.reshape(-1)
                df[gabor_label] = filtered_img
                num += 1           
# CANNY EDGE
edges = cv2.Canny(img, 100,200)
edges1 = edges.reshape(-1)
df['Canny Edge'] = edges1

from skimage.filters import roberts, sobel, scharr, prewitt

# ROBERTS EDGE
edge_roberts = roberts(img)
edge_roberts1 = edge_roberts.reshape(-1)
df['Roberts'] = edge_roberts1

# SOBEL
edge_sobel = sobel(img)
edge_sobel1 = edge_sobel.reshape(-1)
df['Sobel'] = edge_sobel1

# SCHARR
edge_scharr = scharr(img)
edge_scharr1 = edge_scharr.reshape(-1)
df['Scharr'] = edge_scharr1

# PREWITT
edge_prewitt = prewitt(img)
edge_prewitt1 = edge_prewitt.reshape(-1)
df['Prewitt'] = edge_prewitt1

# GAUSSIAN with sigma=3
from scipy import ndimage as nd

gaussian_img = nd.gaussian_filter(img, sigma=3)
gaussian_img1 = gaussian_img.reshape(-1)
df['Gaussian s3'] = gaussian_img1

# GAUSSIAN with sigma=7
gaussian_img2 = nd.gaussian_filter(img, sigma=7)
gaussian_img3 = gaussian_img2.reshape(-1)
df['Gaussian s7'] = gaussian_img3

# MEDIAN with sigma=3
median_img = nd.median_filter(img, size=3)
median_img1 = median_img.reshape(-1)
df['Median s3'] = median_img1

# VARIANCE with size=3
variance_img = nd.generic_filter(img, np.var, size=3)
variance_img1 = variance_img.reshape(-1)
df['Variance s3'] = variance_img1

######################################                

# Now, add a column in the data frame for the Labels
# For this, we need to import the labeled image
labeled_img = cv2.imread('images/Train_masks/Sandstone_Versa0000.tif')
# Remember that you can load an image with partial labels 
# But, drop the rows with unlabeled data

labeled_img = cv2.cvtColor(labeled_img, cv2.COLOR_BGR2GRAY)
labeled_img1 = labeled_img.reshape(-1)
df['Labels'] = labeled_img1

#########################################################

# Define the dependent variable that needs to be predicted (labels)
Y = df["Labels"].values

# Define the independent variables
X = df.drop(labels = ["Labels"], axis=1) 

# Split data into train and test to verify accuracy after fitting the model. 
from sklearn.model_selection import train_test_split
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.4, random_state=20)
1
2
3
4
5
from sklearn.svm import LinearSVC

model = LinearSVC(max_iter=100)
model.fit(X_train, Y_train)
prediction_test = model.predict(X_test)
C:\Users\gzjzx\anaconda3\lib\site-packages\sklearn\svm\_base.py:1225: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
  warnings.warn(

SVM 的速度要比 随机森林 慢得多

1
2
3
from sklearn import metrics

print("Accuracy=", metrics.accuracy_score(Y_test, prediction_test))
Accuracy= 0.9525666606203519

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# You can store the model for future use. In fact, this is how you do machine elarning
# Train on training images, validate on test images and deploy the model on unknown images. 
# 储存训练模型

import pickle

# Save the trained model as pickle string to disk for future use
filename = "sandstone_model"
pickle.dump(model, open(filename, 'wb'))

# To test the model on future datasets
loaded_model = pickle.load(open(filename, 'rb'))
result = loaded_model.predict(X)

# 将分割结果以图片显示
segmented = result.reshape((img.shape))

from matplotlib import pyplot as plt

plt.imshow(segmented, cmap ='jet')
plt.imsave('segmented_rock_RF_100_estim.jpg', segmented, cmap ='jet')

png

虽然这并不意味着支持向量机没用，但我的意思是，对于像素分割，支持向量机可能不是正确的选择，但是如果你有图像分类，支持向量机实际上做得很好。

Icon
Icon
Icon
Icon
上一篇：DIP-Python tutorials for image processing and machine learning(69)-BOVW
下一篇：DIP-Python tutorials for image processing and machine learning(59-67)-Random Forest Classifier